• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    Method for producing (1) a three-dimensional deformable model of an element from an initial base of examples (2) of such elements provided with data for determining, for each of the elements of the initial base , a three-dimensional mesh surface based on points and a triangular network connecting said points, wherein:
    公开号:FR3051951A1
    申请号:FR1654765
    申请日:2016-05-27
    公开日:2017-12-01
    发明作者:Slim Ghorbal；Renaud Seguier；Xavier Bonjour
    申请人:3d Sound Labs；
    IPC主号:
    专利说明:

    Method for producing a three-dimensional deformable model of an element, and associated system
    The present invention relates to a method and system for producing a three-dimensional deformable model of an element.
    Introduced for the first time by Blanz & Vetter 1999 (Volker Blanz and Thomas Vetter's "A Morphable Model for the Synthesis of 3d Faces", In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH '99, pages 187-194, New York, NY, USA, 1999, ACM Press / Addison-Wesley Publishing Co.), deformable models have been gaining popularity ever since.
    These deformable models are also used in three-dimensional or 3D animation (Bao-Cai Yin's "Mpeg-4 compatible 3D facial animation based on morphable model", Cheng-Zhang Wang, Qin Shi, and Yan-Feng Sun, In Machine Learning and Cybernetics, 2005, Proceedings of 2005 International Conference on, Volume 8, pages 4936-4941, Volume 8, Aug 2005, and "Statistical Generation of 3D Facial Animation Models" by Rudomin, A. Bojorquez, and H. Cuevas, In Shape Modeling International, 2002, Proceedings, pages 219-226, 2002.) for the purpose of recognition or identity verification ("Face recognition based on fitting a 3D morphable model" by Volker Blanz and Thomas Vetter, Pattern Analysis and Machine Intelligence, IEEE Transactions on, 25 (9): 1063-1074, 2003; "Automatic 3D face verification from range data" by Gang Pan, Zhaohui Wu, and Yunhe Pan In Acoustics, Speech, and Signal Processing, 2003, Proceedings (ICASSP Ό3) 2003 IEEE International Conference on, volume 3, pages III -193-6 vol.3, April 2003; "Audio- and Video-Based Biometrics Person Authentication" by Alexander M. Bronstein, Michael M. Bronstein, and Ron Kimmel, 4th International Conference, AVBPA 2003 Guildford, UK, June 9-11, 2003 Proceedings, chapter Expression- Invariant 3D Face Recognition, pages 62-70, Springer Berlin Fleidelberg, Berlin, Heidelberg 2003; "Pattern-based face recognition using automatically registered facial surfaces" by MO Irfanoglu, B. Gokberk, and L. Akarun, in Pattern Recognition, 2004, ICPR 2004, Proceedings of the 17th International Conference on, volume 4, pages 183-186 Vol.4, Aug 2004.
    Initially applied to the modeling of faces, these models were gradually transferred to many other elements such as the ears (Chen Li's novel novel ear rectification method using a single image, Zhichun Mu, Feng Zhang, and Shuai Wang , in Intelligent Control and Automation (WCICA), 2012, World Congress on, pages 4891-4896, IEEE, 2012; John D Bustard and Mark S Nixon, "3D morphable model construction for robust ear and face recognition", in Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pages 2582-2589, IEEE, 2010.), the human body as a whole (Brett Allen's "The Space of Human Body Shapes: Reconstruction and Parameterization of Range Scans") , Brian Curless, and Zoran Popovic, in ACM transactions on graphs (TOG), volume 22, pages 587-594, ACM, 2003.), or even animal skeletons (Lionel's "Morphable model of quadrupeds skeletons for animating 3D animais" Reveret, Laurent Favreau, Christine Depraz , and Marie-Paule Cani, in Proceedings of the 2005 ACM SIGGRAPH / Eurographics Symposium on Computer Animation, SCA Ό5, pages 135-142, New York, NY, USA, 2005. ACM.). Nevertheless, whatever the subjects of study, the stages of construction remain substantially identical, namely: 1) Acquisition of 3D data serving as examples of statistical learning. 2) dense matching (also called "dense registration" in English) of said learning examples. 3) Creation of a vector space specific to the study subject by the use of a statistical analysis method such as PCA (Principal Component Analysis), ACI (Independent Component Analysis) or their derivatives .
    The last step 3) of this process gives rise in particular to a so-called mean vector and modes of deformation, the linear combinations of which make it possible not only to reform the learning examples but also to generate new elements (new ones). faces in the case of a deformable model of faces for example).
    However, despite the apparent simplicity of such a method, its implementation must solve two important problems: determining which points can be matched in each learning example and performing this association on a sufficient number of points (typically several thousands). To this end, Blanz and Vetter have proposed the use of an algorithm of flow or optical flow or "Optical Flow Algorithm" in English ("Hierarchical motion-based frame rate conversion" of James R Bergen and R Hingorani, Technical report David Sarno Research Center, 1990). At this point, it should be noted that the laser used for scanning acquisitions, or "scanning" in English, provided a cylindrical representation (also called 2.5 D). Thus, a two-dimensional or 2D image of the texture was immediately available and used for the implementation of the aforementioned algorithm.
    However, in addition to being very sensitive to its initialization, this algorithm requires that the deformations from one example to the other are weak (like successive images of a video), which has no problem. reason to be in the general case. Moreover, the cylindrical representations have the major disadvantage of being generating occlusions. While these are relatively rare in the case of faces, making the Blanz and Vetter method usable, it is different in the case of more complex forms, such as ears, for which the loss of information can be prohibitive.
    Chen Li and associates, for their part, take advantage of the particular form of their subject of study, in this case the ear, and data available, in this case a photo and a depth map of the ear profile view, to develop the hierarchical growth algorithm of mesh triangles or "Triangle Mesh Hierarchical Growth" in English language ("A novel 3D ear reconstruction method using a single image" by Chen Li, Zhichun Mu, Feng Zhang, and Shuai Wang, in Intelligent Control and Automation (WCICA), 2012 World Congress on, pages 4891-4896, IEEE, 2012). A depth map, also called a 2.5 D image or "z map" is a pixel-based z coordinate image, which is usually created using a 3-D camera. The gray levels in the depth map represents the height values.
    An edge detection is performed on the photo and two initial marks are placed by the operator. The intersection of the mediator of the segment connecting these two points with the outer contour of the ear creates a third point. By iterating this process from the new point and precedents, the authors create 17 descriptive points of the outer contour of the ear. By a similar path, they also create other series of descriptive points on the inner contours.
    Then, a series of triangulations allows them to obtain a deterministic division of the ear in 23552 triangles and 13601 points. Assuming the camera and the camera used to make the depth map positioned at the same location, 3D coordinates can be associated with the splitting performed, thus completing the mapping. Nevertheless, the very nature of the data renders inaccessible the convolutions of the ears and allows the final only the obtaining of a simplified modelization, limiting in fact the potential applications which can result from it.
    In addition, the matching method based on global and non-local geometric considerations, such as the intersection of a line from one end of the image to a curve at the other end, dilutes or even loses totally the semantic information conveyed by the image.
    Also, characteristic points of the ear such as tragus or anti-tragus can not be associated with one or more of the constructed descriptive points.
    Finally, as presented by its authors, this method has the important disadvantage of giving correct results only to convex forms, and would give instead chaotic results on simple geometries, star or crescent moon for example.
    Kaneko et al. (Shoken Kaneko's Ear shape modeling for 3D audio and acoustic Virtual Reality: The shapebased average hrtf ", Tsukasa Suenaga, Fujiwara Mai, Kazuya Kumehara, Futoshi Shirakihara, and SaSatoshi Sekine, Audio Engineering Society Conference: 61 st International Conference : Audio for Games, Audio Engineering Society, 2016.) uses scans or X-ray scans of volunteer ear molds and favors the use of non-rigid 3D mapping methods ("A new point Haili Chui and Anand Rangarajan, Computer Vision and Image Understanding, 89 (2): 114-141, 2003; Bing Jian and Baba C Vemuri Pattern, "Robust point set registration by gaussian mixture models" Analysis and Machine Intelligence, IEEE Transactions on, 33 (8): 1633-1645, 2011.). The meshes are composed of about 3000 vertices and the vectors of deformation transforming a mesh of reference in the others of the base are sought with the aid of mixtures of Gaussians.
    According to one aspect of the invention, there is provided a method for generating a three-dimensional deformable model of an element from an initial basis of examples of such elements provided with data for determining, for each of the elements of the initial base, a three-dimensional mesh surface based on points and a triangular network connecting said points, wherein: for each element example of the initial base, at each point of its mesh surface, determining (by measurement or calculation) the value of at least one parameter representative of the shape of the surface of the element at this point, to obtain an improved basis of element examples; for each element example of the improved base, corresponding to the elements of the initial base, the mesh surface is flattened in three dimensions to obtain a two-dimensional representation of said mesh surface; matching, on the set of two-dimensional representations of the mesh surfaces of said elements, a plurality of respective points using said determined values of the parameter or parameters representative of the shape of the surface of the element at said points and a method analyzing said two-dimensional representations of the meshed surfaces; subsampling, from said mapped points, said three-dimensional meshed surfaces of the initial base; determining a model of the element from the three-dimensional meshed surfaces of the initial base including a mean shape of the element and modes of deformation; and remeshing said average form of the element.
    Thus, the invention is not dependent on the presence of texture information and can handle data sets that are devoid of such data, such as MRI results.
    According to one embodiment, said one or more parameters representative of the shape of the surface of the element at a point of the mesh surface of an example of the initial base comprise a local curvature at said point and / or a descriptor of form at that point.
    Thus, it is possible to vary the complexity of the one or more parameters representative of the shape of the surface of the element at a point on the surface according to the needs and / or the external constraints.
    In one embodiment, said local curvature comprises a minimum curvature and / or a maximum curvature and / or a Gaussian curvature and / or a mean curvature.
    Thus, the choice of the type of curvature is a lever to adapt, according to the needs, to the peculiarities of the object of study.
    According to one embodiment, the shape descriptor comprises a shape correction surface patch histogram or SPHIS for the acronym "Surface Patch Histogram of Index Shape" in the English language.
    Thus, the method may be configured to detect the more or less pronounced presence of one or more types of shapes rather than merely measuring the curvature.
    For example, said flattening uses an ABF, LSCM, ABF ++, or HLSCM method.
    Thus, the method is not constrained by the use of a particular method of flattening but can select one method or another depending on the benefits provided and the needs of the moment.
    According to one embodiment, said mapping uses a division of the two-dimensional representations in Ne bending levels distributed uniformly over the range of values taken by the values of the parameter or parameters representative of the shape of the surface of the item.
    Thus, two-dimensional representations can be segmented according to objective and reproducible criteria.
    As a variant, said mapping uses a division of the two-dimensional representations into Ne bending levels taking into account the statistical distribution of the values taken by the values of the parameter or parameters representative of the shape of the surface of the element.
    Thus, it is possible to make the previous variant independent of the presence of extreme values but also to take into account the over-representation or under-representation of certain ranges of values.
    In one embodiment, which of said mappings uses a number of manually mapped points.
    Thus, the operator precisely controls their positioning. This is particularly useful when building small models (for testing purposes or for lack of learning examples).
    For example, the method is semi-automatic, and as said mapping is made, the number of manually mapped points of the current element decreases with the number of elements processed.
    Thus, the construction time of the model and its cost in human resources are rendered marginal when adding new examples.
    In a variant, the method is automatic based on active contours, and the number of points mapped manually is zero.
    Thus, the construction time of the model and its cost in human resources are minimized, making it easier to optimize the other construction parameters (curve calculation parameters, number of points of the training examples, choice of flattening algorithm, etc.). These last two variants are particularly interesting in the context of important learning bases.
    In one embodiment, said element is a right ear and / or a left ear, and / or the head, and / or the torso of an individual.
    According to another aspect of the invention, there is also provided a system for developing a three-dimensional deformable model of an element from an initial basis of examples of such elements provided with data for determining, for each of the elements of the initial base, a three-dimensional mesh surface based on points and a triangular network connecting said points, comprising a computer configured to implement the method as previously described. The invention will be better understood from the study of some embodiments described by way of non-limiting examples and illustrated by the appended drawings in which: FIGS. 1 to 7 schematically illustrate a method according to one aspect of the applied invention; to human ears; and - Figures 8 to 10 schematically illustrate a method according to one aspect of the invention applied to human faces.
    The present invention is an alternative to the aforementioned methods and allows the creation of a deformable model of any type of subject or element from the study of its morphology.
    In the remainder of the description, the described examples of elements will be human ears or faces, but the invention may apply to any other element.
    In particular, the present invention does not require any texture information and thus effectively avoids the pose and illumination problems experienced by the flow or optical flow algorithms, such as the algorithms called structure from motion or SFM for acronym for "structure from motion" in English or algorithms called structure from shadows or SES for acronym of "structure from shading" in English. In addition, the invention makes it possible to adapt naturally to data in three dimensions or 3D as those in 2.5D. Finally, the present invention makes it possible to preserve the semantic information, or in other words to preserve the physical meaning conveyed by a zone, a group of vertices or even a single vertex. Thus, on the example of a human face, the vertices composing the nose of the middle form will also compose the nose of any face of the model after deformation. This observation is also valid for substructures as in the present case: the tip of the nose, the right and left nostrils or the ridge.
    Figure 1 shows the major steps of the method according to one aspect of the invention.
    In other words, FIG. 1 illustrates a method 1 for producing a three-dimensional deformable model of an element from an initial base of examples, loaded 2 in the computer means implementing the method. such elements provided with data for determining, for each of the elements of the initial base, a three-dimensional mesh surface based on points and a triangular network connecting said points, in which: for each element example of the initial basis, at each point of its mesh surface, the value of at least one parameter representative of the shape of the surface of the element at this point is determined (by measurement or calculation) to obtain an improved base of examples of elements; for each element example of the improved base, corresponding to the elements of the initial base, flattening or unfolding of the mesh surface in three dimensions is performed to obtain a two-dimensional representation of said mesh surface; 5, on the set of two-dimensional representations of the meshed surfaces of said elements, a plurality of respective points are mapped using said determined values of the parameter or parameters representative of the shape of the surface of the element at said points and a method of analyzing said two-dimensional representations of the meshed surfaces; subsampling 6, from said mapped points, said three-dimensional meshed surfaces of the initial base; determining a model 7 of the element from the three-dimensional meshed surfaces of the initial base comprising a mean shape of the element and deformation modes; and remeshing 8 of said average shape of the element.
    Each sample element of the initial database can be subsampled.
    Thus, when the available computing power is limited, it is possible to adapt the data accordingly.
    Alternatively, one can subsample each example element of the initial database except one of said examples taken for reference.
    Thus, it is possible to improve the efficiency of the automatic matching step without requiring significantly more computing power.
    Figures 2a and 2b show an example of a model obtained from an ear base. In Figure 2a is shown the non-meshing middle ear, and in Figure 2b is shown the same deformed ear according to the third mode of deformation. In this case it is about straight ears.
    More precisely, the models obtained by principal component analysis, of acronym ACP, are in the form of a mean and deformation modes or eigenvalue / eigenvector pairs ranked in order of importance, or in others. terms in descending order of eigenvalues. We can therefore speak of the first mode of deformation, the second mode of deformation, etc ...
    In the present invention, in the 3D universe, each mode of deformation represents a set of displacements type undergone by the elements of the point cloud. It is possible to see these types of displacement as the data of a direction and a speed of movement for each point. The data of a multiplicative coefficient, which could be assimilated to a duration in the previous analogy, makes it possible to calculate the exact displacement.
    In Figure 2a is shown the middle ear without deformation and the previous one is the same ear with deformations. In both cases, the gray level represents the displacement of the points relative to their position within the middle ear. In Figure 2a, since there is no movement, the gray is uniform.
    In Figure 2b, the third mode of deformation has been used. It is the highest greyscale vertices that have moved the most and vice versa.
    The present model thus makes it possible to highlight physical substructures of the ear which tend to evolve together (or on the contrary separately if one works by complementarity).
    The gray level of each point is associated with its deviation from its position in the average shape (the higher the gray level, the larger the difference).
    The implementation of the method according to one aspect of the invention is carried out as follows: 1) A base of training examples is supposed to be available, each of the examples allowing, directly or after treatments, the reconstruction of a surface mesh in R ^. 2) For each example we carry out: - A measurement of the local geometrical characteristics in each point of each example. The result of each measurement is associated with the point used for its realization.
    This measurement can be summarized as a local curvature, as shown in Figure 3, such as the minimum, maximum, Gaussian or average curvature at the point considered, to a more complex shape descriptor such as an index surface patch histogram. form or SPHIS for acronym for "Surface Patch Histogram of Index Shape" in English, or an equivalent or a combination of the above. The 3D ear of Figure 3 at gray levels that depend on the local mean curvature, the higher the local average curvature, the darker the gray is. An unfolding of the surface which thus makes it possible to obtain a representation of each mesh in the form of a 2D image, as illustrated in FIGS. 4a and 4b, respectively corresponding to the left and right ears of subject 9 after calculation of curvature and flattening, noted im2D and a connected graph, classically noted Gc but not shown. In FIGS. 4a and 4b, the gray levels are such that the greater the local average curvature, the darker is the gray.
    This unfolding, also called "unwrapping" in English, can be done in many ways, as with flattening algorithms based on angles, with ABF acronym for "Angle-based Flattening" in English, algorithms of LSCM acronym LSCM acronym for Least Square Conformai Maps in English, or their derivatives (ABF ++, FISLCM for Flierarchical Least Square Conformai Maps in English, ...). 3) A mapping of a maximum of points is performed from 2D images using the characteristics measured in point 2) and the analysis methods related to 2D image processing, as shown in Figure 5. On Figure 5 are shown on the left in 2D and on the right in 3D the same ear after manual mapping of 88 points according to the isocurbures 9, which are materialized on the left 2D image. In Fig. 5, the gray levels are such that the higher the average local curvature, the darker the gray. 4) The points retained during the matching are then used to downsample the initial 3D meshes. The resulting point clouds are then used to build the actual model using conventional construction tools such as principal component analysis, acronym ACP or intermediate component analysis, with the acronym AGI. .).
    We then obtain a so-called average shape and deformation modes, as shown in Figure 2b. 5) A remeshing of the average shape gives a surface to the model.
    Here follows an embodiment of the method of the invention, relating to a deformable 3D ear model.
    The base used consists of the ten freely accessible examples of the SYMARE database for "Sydney-York Morphological And Recording of Ears" in English.
    In addition, all the left ears of these ten pairs of ears were symmetrized with respect to the sagittal plane so as to have twenty straight ears (the ten initial lines and the ten straight lines coming from the symmetrization of the ten left).
    We denote 1 = [l; 20] the set of indices of these right ears and i = 1 the index of the right ear taken as reference right ear.
    For reasons of coherence, the meshes of the ears thus obtained are subsampled to about 6900 vertices. This step, purely optional, is present in order to optimize the digital processing times and to facilitate the subsequent integration of possible other learning examples.
    Finally, the left ear of the first subject of the base was chosen as reference ear after symmetrization in right ear. In the rest of the document, all the notation indexed by ref refer naturally to this reference (in particular, we have iref = 1).
    Point 2) of the description of the invention is then carried out. The local mean curvature was retained as a geometric feature and applied as texture to the 3D meshes, as shown in Figure 3.
    The unfolding was achieved by the LSCM LSCM algorithm for "Least Square Conformai Maps" in English, as illustrated in Figures 4a and 4b.
    As stated in point 3), other algorithms are employable. There are no special requirements. The mapping of the vertices of the connected graphs required the following steps: 1) Trimming of the 2D images according to Ne = 10 levels of curvature evenly distributed over the range of values taken by the curvature measurements, as illustrated in FIG. 2) Selection of = 88 vertices of the connected graph of reference.
    This selection is made by following the line of ric isocurbure lines, as illustrated in the example of Figure 6, ric ε themselves chosen to ensure a uniform distribution of vertices. 3) Selection of the corresponding vertices in each connected graph (not represented) corresponding G ^, ie l {iref} · 4) Realization of a triangulation (in this case, of Delaunay) on the fjmanu vertices coming from the connected graph of the indexed reference ear iref and transfer of the connected connectivity graph or connected graph to the other groups of previously isolated vertices.
    This creates a finite set of triangles, which is indexed by / c N. In this case, / = [1,163].
    We notice
    the assembly thus created for the i® "^ ® ear, as shown in Figure 7. In Figure 7, are represented the indexed ears 1 at the top and 9 at the bottom, respectively unfolded in 2D on the left and in 3D on the right A triangulation of the selected vertices was carried out on the ear 1 and transferred on the ear 9. By way of example, the triangle number 116 of this triangulation is highlighted on the two ears 1 and 9. 5) For every I J, we consider
    the set of j ^ "^ ®® triangles Each element tj j potentially contains vertices of the graph G ^, whose barycentric coordinates are calculated in the reference proper to the triangle t (6). included in and, for each of them, one looks in every other triangle
    the nearest vertex in terms of barycentric coordinates. This creates new mappings.
    However, since one vertex of a graph does not have to be associated with several vertices of another, the least interesting conflicting mappings from the point of view of the barycentric distance are eliminated.
    We thus obtain a set of automatic correspondences starting from the elements of 7}, j g /.
    This being done for all j g /, we finally get g N automatic matches for all ears. In this example,
    points.
    Among the possible improvements and variants, we can list the following: - The use of other criteria for characterization of local geometry that the only curvature, including the joint use of several of them. - Over-sampling of all ears or all but the reference one. This makes it possible to increase the efficiency of the automatic matching and, consequently, the final resolution of the model. This additional step allows to go from 1460 points to 5076, to put in front of 6900 summits of the initial meshes. For example, this oversampling can use the barycenters of the initial triangles. - The parameters Ν ^. and can of course be attached to other values. - The curvature levels may not be uniformly distributed over the available range but take into account the statistical distribution of curvature values. - The points selected in step 2) of the matching can be automatically or semi-automatically, for example: - by making a gradual learning of the selection process on the first ears allowing a preselection on the following ears. - using methods based on active contours or "snakes" in English to establish the details of the transformations of an isoline of an image to its corresponding of another image.
    In another embodiment which follows, the method is applied to human faces.
    The face database used in this example consists of Examples 2, 5, 6 and 14 of the UWA face database 3D database of the University of Western Australia (UWA), available at following address: "http://staffhome.ecm.uwa.edu.au/~00053650/databases.html".
    As for the previously described example applied to human ears, a mesh has been chosen as a reference. In this case, that of the subject number 2 of the database.
    Similarly, the local mean curvature was also retained as a geometric feature and applied as a texture to the 3D meshes, as shown in Figure 8, where the gray levels are representative of the local mean curvature plus gray level. is high or dark, the higher the average local curvature.
    On the other hand, unlike in the previous case, the available meshes have not been subsampled. The number of initial peaks varies between 16655 and 25951. Step 2 of the description of the invention is then carried out following the same methodology as for the example described based on ears.
    The only notable differences are the number of learning examples (4 faces) and the number of manually annotated points (Nmanu) as illustrated in FIG. 9 which represents an example of 2D and 3D representation of the same face after implementation. manual correspondence of 37 points according to the isocurbures, like the figure 5 for the example of the ears.
    The result obtained is a 6846 vertex deformable face model consisting of a middle face, as shown in Figure 10 in gray levels, accompanied by 3 deformation modes.
    In the present application, it has been explained an innovative way of constructing a deformable model. This method has many advantages over the state of the prior art, including: - A potential application to any type of 3D form, without loss of information. - The preservation of the semantic aspect attached to the object of study, thanks to the local character of the considered characteristics. - The possibility of being fully automated. - A frankness vis-à-vis the texture of the learning examples and thus the illumination conditions during the acquisition of the data. - The ability to incorporate 2D image analysis algorithms, many more and more mature than those of the 3D world.
    The steps of the method described above may be performed by one or more programmable processors executing a computer program for performing the functions of the invention by operating on input data and generating output data.
    A computer program can be written in any form of programming language, including compiled or interpreted languages, and the computer program can be deployed in any form, including as a stand-alone program or as a subroutine, element or other unit suitable for use in a computing environment. A computer program can be deployed to run on one computer or multiple computers at a single site or spread across multiple sites and interconnected by a communications network.
    The preferred embodiment of the present invention has been described. Various modifications can be made without departing from the spirit and scope of the invention. Therefore, other implementations are within the scope of the following claims.
    权利要求:
    Claims (12)
    [1" id="c-fr-0001]
    A method of producing (1) a three-dimensional deformable model of an element from an initial base of examples (2) of such elements provided with data for determining, for each of the elements of the initial base, a three-dimensional mesh surface based on points and a triangular network connecting said points, wherein: for each element example of the initial base, at each point of its meshed surface, is determined (3) the value of at least one parameter representative of the shape of the surface of the element at this point, to obtain an improved basis of element examples; for each element example of the improved base corresponding to the elements of the initial base, flattening (4) of the mesh surface in three dimensions is performed to obtain a two-dimensional representation of said mesh surface; matching (5), on the set of two-dimensional representations of the meshed surfaces of said elements, a plurality of respective points using said determined values of the one or more parameters representative of the shape of the surface of the element at said points and a method of analyzing said two-dimensional representations of the meshed surfaces; subsampling (6), from said mapped points, said three-dimensional meshed surfaces of the initial base; determining (7) a model of the element from the three-dimensional meshed surfaces of the initial base including a mean shape of the element and modes of deformation; and remeshing (8) said average shape of the element.
    [2" id="c-fr-0002]
    The method of claim 1, wherein said one or more parameters representative of the shape of the surface of the element at a point of the mesh surface of an example of the initial base comprise a local curvature at said point and / or a form descriptor at that point.
    [3" id="c-fr-0003]
    The method of claim 2, wherein said local curvature comprises a minimum curvature and / or a maximum curvature and / or a Gaussian curvature and / or a mean curvature.
    [4" id="c-fr-0004]
    The method according to claim 2 or 3, wherein the shape descriptor comprises a shape correction surface patch histogram or SPHIS for the acronym "Surface Patch Histogram of Index Shape" in English.
    [5" id="c-fr-0005]
    5. Method according to one of the preceding claims, wherein said flattening (4) uses an ABF method, LSCM, ABF ++, or HLSCM.
    [6" id="c-fr-0006]
    The method according to one of claims 1 to 5, wherein said mapping (5) uses a split of the two-dimensional representations into non-uniformly distributed curvature levels over the range of values taken by the values of the parameters representative of the shape of the surface of the element.
    [7" id="c-fr-0007]
    7. Method according to one of claims 1 to 5, wherein said mapping (5) uses a division of two-dimensional representations into Ne bending levels taking into account the statistical distribution of the values taken by the values of the parameters representative of the shape of the surface of the element.
    [8" id="c-fr-0008]
    The method according to one of the preceding claims, wherein said mapping (5) uses a number of manu points mapped manually.
    [9" id="c-fr-0009]
    9. The method of claim 8, semi-automatic, wherein as and when said mapping (5), the number of points manually matched the current item decreases with the number of elements treated .
    [10" id="c-fr-0010]
    10. The method of claim 8, automatic based active contours, wherein the number of points manually matched is zero.
    [11" id="c-fr-0011]
    11. Method according to one of the preceding claims, wherein said element is a right ear and / or a left ear, and / or the head, and / or the torso of an individual.
    [12" id="c-fr-0012]
    12. System for developing a three-dimensional deformable model of an element from an initial basis of examples of such elements provided with data for determining, for each of the elements of the initial base, a mesh surface three-dimensional point-based and a triangular network connecting said points, comprising a computer configured to implement the method according to one of the preceding claims.
    类似技术:
    公开号 | 公开日 | 专利标题
    FR3051951A1|2017-12-01|METHOD FOR PRODUCING A DEFORMABLE MODEL IN THREE DIMENSIONS OF AN ELEMENT, AND SYSTEM THEREOF
    Yamaguchi et al.2018|High-fidelity facial reflectance and geometry inference from an unconstrained image
    Huang et al.2018|Deep volumetric video from very sparse multi-view performance capture
    KR102146398B1|2020-08-20|Three dimensional content producing apparatus and three dimensional content producing method thereof
    KR101893047B1|2018-08-29|Image processing method and image processing device
    JP2018073393A|2018-05-10|3d reconstruction of real object from depth map
    JP4925097B2|2012-04-25|How to generate a bilinear model of a face
    Oxholm et al.2014|Multiview shape and reflectance from natural illumination
    JP4780198B2|2011-09-28|Authentication system and authentication method
    Muratov et al.2016|3DCapture: 3D Reconstruction for a Smartphone
    Cao et al.2018|Sparse photometric 3D face reconstruction guided by morphable models
    EP2724203B1|2017-09-13|Generation of map data
    Wang et al.2016|Light field imaging based accurate image specular highlight removal
    JP2014149677A|2014-08-21|Makeup support apparatus, makeup support system, makeup support method and makeup support system
    Seck et al.2016|Ear-to-ear capture of facial intrinsics
    Berrier et al.2015|Improved appearance rendering for photogrammetrically acquired 3D models
    Ferková et al.2020|Age and gender-based human face reconstruction from single frontal image
    WO2006117374A2|2006-11-09|Method for three-dimensionally reconstructing an articulated member or a set of articulated members
    Golyanik et al.2017|A framework for an accurate point cloud based registration of full 3d human body scans
    EP3614306A1|2020-02-26|Method for facial location and identification and pose determination, from a three-dimensional view
    van Dam et al.2016|Face reconstruction from image sequences for forensic face comparison
    FR3057981B1|2019-07-26|METHOD FOR PRODUCING A 3D POINT CLOUD REPRESENTATIVE OF A 3D EAR OF AN INDIVIDUAL, AND ASSOCIATED SYSTEM
    FR3065097A1|2018-10-12|AUTOMATED METHOD FOR RECOGNIZING AN OBJECT
    KR102187143B1|2020-12-04|Three dimensional content producing apparatus and three dimensional content producing method thereof
    Neuhaus et al.2018|High-resolution hyperspectral ground mapping for robotic vision
    同族专利:
    公开号 | 公开日
    WO2017202634A1|2017-11-30|
    CN109844818A|2019-06-04|
    US10762704B2|2020-09-01|
    US20200118332A1|2020-04-16|
    FR3051951B1|2018-06-15|
    US10489977B2|2019-11-26|
    US20190147650A1|2019-05-16|
    EP3465629A1|2019-04-10|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US7756325B2|2005-06-20|2010-07-13|University Of Basel|Estimating 3D shape and texture of a 3D object based on a 2D image of the 3D object|
    KR101624808B1|2011-08-09|2016-05-26|인텔 코포레이션|Parameterized 3d face generation|
    KR102285376B1|2015-12-01|2021-08-03|삼성전자주식회사|3d face modeling method and 3d face modeling apparatus|
    FI20165211A|2016-03-15|2017-09-16|Ownsurround Ltd|Arrangements for the production of HRTF filters|
    US10474883B2|2016-11-08|2019-11-12|Nec Corporation|Siamese reconstruction convolutional neural network for pose-invariant face recognition|
    US10600238B2|2017-03-09|2020-03-24|Institute Of Automation, Chinese Academy Of Sciences|Image tampering forensics method and apparatus|US10805757B2|2015-12-31|2020-10-13|Creative Technology Ltd|Method for generating a customized/personalized head related transfer function|
    SG10201800147XA|2018-01-05|2019-08-27|Creative Tech Ltd|A system and a processing method for customizing audio experience|
    US10390171B2|2018-01-07|2019-08-20|Creative Technology Ltd|Method for generating customized spatial audio with head tracking|
    US10966046B2|2018-12-07|2021-03-30|Creative Technology Ltd|Spatial repositioning of multiple audio streams|
    US11221820B2|2019-03-20|2022-01-11|Creative Technology Ltd|System and method for processing audio between multiple audio spaces|
    法律状态:
    2017-04-27| PLFP| Fee payment|Year of fee payment: 2 |
    2017-12-01| PLSC| Search report ready|Effective date: 20171201 |
    2018-05-01| PLFP| Fee payment|Year of fee payment: 3 |
    2019-05-10| PLFP| Fee payment|Year of fee payment: 4 |
    2020-05-26| PLFP| Fee payment|Year of fee payment: 5 |
    2020-07-10| TP| Transmission of property|Owner name: MIMI HEARING TECHNOLOGIES GMBH, DE Effective date: 20200602 |
    2021-04-26| PLFP| Fee payment|Year of fee payment: 6 |
    优先权:
    申请号 | 申请日 | 专利标题
    FR1654765|2016-05-27|
    FR1654765A|FR3051951B1|2016-05-27|2016-05-27|METHOD FOR PRODUCING A DEFORMABLE MODEL IN THREE DIMENSIONS OF AN ELEMENT, AND SYSTEM THEREOF|FR1654765A| FR3051951B1|2016-05-27|2016-05-27|METHOD FOR PRODUCING A DEFORMABLE MODEL IN THREE DIMENSIONS OF AN ELEMENT, AND SYSTEM THEREOF|
    CN201780032523.6A| CN109844818A|2016-05-27|2017-05-15|For establishing the method and associated system of the deformable 3d model of element|
    PCT/EP2017/061607| WO2017202634A1|2016-05-27|2017-05-15|Method for establishing a deformable 3d model of an element, and associated system|
    EP17723989.4A| EP3465629A1|2016-05-27|2017-05-15|Method for establishing a deformable 3d model of an element, and associated system|
    US16/300,044| US10489977B2|2016-05-27|2017-05-15|Method for establishing a deformable 3D model of an element, and associated system|
    US16/656,993| US10762704B2|2016-05-27|2019-10-18|Method for establishing a deformable 3D model of an element, and associated system|
    [返回顶部]